Database search algorithms are used in many types of systems. For example, biometric search algorithms are used to determine whether or not the fingerprints found at a crime scene match those in a biometric repository, web search algorithms are used to find web pages relevant to a user query, etc.
In general, search algorithms allow a user to enter a query, such as a key word, and return a set of (i.e., zero, one or more) database records that the algorithm identifies as matching the query. Depending on the database, query and accuracy, a particular algorithm may or may not return records that actually match the query. Because there are numerous different types of search algorithms, providing a process for evaluating search algorithms is useful for selecting a search algorithm that will best optimize search performance.
Evaluations of the accuracy of database search algorithms are typically performed using a sample of the database and a sample of the searches. Samples are used because it would be too time consuming and costly to evaluate accuracy for all searches of the entire database. The known solutions to this problem all involve selecting a sample of searches and database records that have been identified by some method as being matched to each other. Often this sample is supplemented with additional random samples of searches and database records.
There are at least two issues with the known solutions to this problem. First, the known solutions produce biased results because they select a sample from a pool of queries and database records that have been matched using existing systems. Information retrieval evaluation based on this method is inherently biased against systems that did not contribute to the pool. A second problem with existing solutions is that very large samples are needed to measure very small error rates, such as the False Accept Rate (FAR) of fingerprint matching algorithms. Larger sample sizes make the evaluation of search algorithms more costly and time consuming.